Nexus: worker, workflow-backed operations, and workflow caller #813

dandavison · 2025-04-02T19:06:35Z

Initial Python Temporal Nexus implementation.

Temporal SDK PR to accompany

Nexus SDK PR Initial implementation nexus-rpc/sdk-python#1
samples-python PR Nexus samples samples-python#174

Contains Nexus worker, components for users to define workflow-backed Nexus operations, and the ability to start and cancel a Nexus operation from a workflow.

Notes for reviewers

nexusrpc.handler.start_workflow is a top-level static function, but currently there's still a public contextvar object nexusrpc.handler.temporal_operation_context. Let's settle on approach there. If we're using module-level getters I think they need to be named something like get_client() etc, despite the fact that we have activity.metric_meter() ? Little bit more discussion needed there.
I'll do another cleanup pass on docstrings, API docs, etc.

- fetch result - error handling

…tionContext" This reverts commit 75d16b0.

This reverts commit dec0a60.

cretz

Added another round of review. Note, there are some comments from previous rounds that are still valid/unaddressed.

cretz · 2025-06-27T19:19:15Z

temporalio/exceptions.py

+        self._type = type
+        self._retryable = retryable


To clarify what I'm saying here:

Suggested change

self._type = type

self._retryable = retryable

self.type = type

self.retryable = retryable

We should not hide the attributes from the user, same for operation error which it does appear we added property getters for (I'm fine with either property getters or visible/documented attributes)

cretz · 2025-06-27T19:20:51Z

temporalio/nexus/__init__.py

+from ._operation_context import (
+    _TemporalCancelOperationContext as _TemporalCancelOperationContext,
+)
+from ._operation_context import (
+    _TemporalStartOperationContext as _TemporalStartOperationContext,
+)


Are we intentionally exposing _ prefixed types?

cretz · 2025-06-27T19:23:05Z

temporalio/nexus/_operation_context.py

+):
+    ctx = _try_temporal_context()
+    if ctx is None:
+        raise RuntimeError("Not in Nexus operation context.")


We need to expose a in_handler() or something similarly named akin to in_activity() and in_workflow() that exist today

cretz · 2025-06-27T19:24:27Z

temporalio/nexus/_operation_context.py

+    """The task queue of the worker handling this Nexus operation."""
+
+
+def info() -> Info:


We may need to open issue with other SDKs for this feature, they have no concept of accessible "info" inside handlers

cretz · 2025-06-27T19:25:20Z

temporalio/nexus/_operation_context.py

+        return workflow_handle
+
+
+@dataclass(frozen=True)


IMO not much value making this a dataclass, just a constructor that is marked "don't use" accepting the context and storing as an attr I think is good

cretz · 2025-06-27T20:21:09Z

tests/conftest.py

@@ -99,6 +99,7 @@ def env_type(request: pytest.FixtureRequest) -> str:
 @pytest_asyncio.fixture(scope="session")
 async def env(env_type: str) -> AsyncGenerator[WorkflowEnvironment, None]:
    if env_type == "local":
+        http_port = 7243


Why not use the official way of starting/cancelling Nexus operations via workflows instead of this less-official HTTP manner? Is there something that HTTP gives you that Nexus client in the workflow doesn't that you need to test? EDIT: I see we have another thread at #813 (comment) that overlaps this from before, can discuss there.

cretz · 2025-06-27T20:24:38Z

tests/helpers/nexus.py

+# TODO(nexus-preview): How do we recommend that users create endpoints in their own tests?
+# See https://github.com/temporalio/sdk-typescript/pull/1708/files?show-viewed-files=true&file-filters%5B%5D=&w=0#r2082549085


Yes, this is important. We need to improve the Nexus testing story. We should consider accepting Nexus endpoints on WorkflowEnvironment.start_local and WorkflowEnvironment.start_time_skipping and maybe even expose a helper creation method on the environment itself for use post-start. We can move some of that into Core or even into the CLI dev server and Java test server, that can remain to be seen.

Also, can you confirm that immediately upon create_nexus_endpoint return the endpoint can be used? In other cases (create namespace, create search attribute) there have been issues in the past with eventual consistency.

cretz · 2025-06-27T20:28:38Z

tests/helpers/nexus.py

+    async def fetch_operation_info(
+        self,
+        operation: str,
+        token: str,
+    ) -> httpx.Response:
+        async with httpx.AsyncClient() as http_client:
+            return await http_client.get(
+                f"{self.server_address}/nexus/endpoints/{self.endpoint}/services/{self.service}/{operation}",
+                # Token can also be sent as "Nexus-Operation-Token" header
+                params={"token": token},
+            )
+
+    async def fetch_operation_result(
+        self,
+        operation: str,
+        token: str,
+    ) -> httpx.Response:
+        async with httpx.AsyncClient() as http_client:
+            return await http_client.get(
+                f"{self.server_address}/nexus/endpoints/{self.endpoint}/services/{self.service}/{operation}/result",
+                # Token can also be sent as "Nexus-Operation-Token" header
+                params={"token": token},
+            )


These two are unused, correct? I don't think these are considered stable/supported at this time

cretz · 2025-06-27T20:31:20Z

temporalio/nexus/_token.py

+
+    namespace: str
+    workflow_id: str
+    _type: OperationTokenType = OPERATION_TOKEN_TYPE_WORKFLOW


Hidden/private fields are confusing on user-facing dataclasses IMO. Can this be removed?

cretz · 2025-06-27T20:36:21Z

tests/nexus/test_handler.py

+    await _test_start_operation(test_case, True, env)
+
+
+async def _test_start_operation(


Not sure it's too late to change. Ideally we start Nexus like a user does, we cancel Nexus like a user does. To use advanced/secret ways to do this makes these tests hard to give to users as examples of how they might test (though that's obviously not the goal of the tests, it's a nice benefit). If something is impossible via workflow caller, we should not be using it since a user can't either.

For the most part, everything we do in a test a user should be able to do in their tests too. It has the added benefit of confirming how a user might test these things and spotting any gaps. If you are unable to use existing pieces to test the user-facing components and have to resort to HTTP, what are we going to tell users when they encounter the same situation and want to test the same thing the same way?

cretz · 2025-06-27T21:18:19Z

temporalio/worker/_nexus.py

+                        # TODO(nexus-prerelease): do we need to track cancel operation
+                        # tasks as we do start operation tasks?


Yes, we do IMO. Cancels can be canceled (i.e. timeout from Core POV)

dandavison force-pushed the nexus branch 2 times, most recently from a845a6f to 7a121bb Compare April 5, 2025 15:08

dandavison force-pushed the nexus branch 10 times, most recently from 862e4f9 to 8ac0192 Compare April 17, 2025 12:17

dandavison force-pushed the nexus branch 4 times, most recently from 8940d51 to 520aecf Compare April 26, 2025 16:16

dandavison force-pushed the nexus branch 2 times, most recently from 2fc971f to 8bd6011 Compare April 29, 2025 13:11

dandavison force-pushed the nexus branch from 18e268a to 2eb967a Compare May 6, 2025 15:55

dandavison changed the title ~~Nexus prototype~~ Nexus May 8, 2025

dandavison force-pushed the nexus branch from 7876ecc to 7fe6868 Compare May 8, 2025 22:07

dandavison force-pushed the nexus branch 2 times, most recently from 5d54f48 to 8ddfb94 Compare May 24, 2025 01:49

dandavison force-pushed the nexus branch 7 times, most recently from fa3e8ec to f7bf47b Compare May 27, 2025 20:34

dandavison added 17 commits June 27, 2025 16:00

Revert disabling of sandbox for nexus workflow tests

ff1ce3b

Passthrough 3rd-party imports in tests helpers module

8847f0a

uv.lock

8d54495

Strengthen warning note

b079bab

Docstrings, comments

3e1d7ba

Type-level cleanup/evolution in workflow caller

f87531f

TODOs

f5b7de8

- fetch result - error handling

Move logger

6483587

Separate Temporal context for each operation verb

b36c20c

Make Temporal context classes non-private

3bf6507

Use TemporalStartOperationContext instead of WorkflowRunOperationContext

ed361ea

Revert "Use TemporalStartOperationContext instead of WorkflowRunOpera…

4b6d343

…tionContext" This reverts commit 75d16b0.

Make WorkflowRunOperationContext subclass StartOperationContext

e5807f0

Mark TemporalStartOperationContext as private

4a36d33

Handle OperationError consistently with HandlerError

dec0a60

Revert "Handle OperationError consistently with HandlerError"

cb4b92d

This reverts commit dec0a60.

RTU: operation_id -> operation_token

99c015b

cretz reviewed Jun 27, 2025

View reviewed changes

Fix cancellation context bug

09e65ff

dandavison force-pushed the nexus branch from ecf5721 to 09e65ff Compare June 27, 2025 20:47

cretz reviewed Jun 27, 2025

View reviewed changes

dandavison added 9 commits June 28, 2025 13:14

RTU: Use nexusrpc.get_service_definition

6feef17

Docstring

3563b2a

RTU get_operation_factory

6fa2e26

Workflow OperationError / HandlerError test

63e6af6

Convert nexus_handler_failure_info as nexusrpc.HandlerError

c05cc6c

RTU: Move HandlerError to root module

1541b85

RTU: test is fixed by syncio.sync_operation

b771ada

RTU: unskip test

c8a4daf

RTU: syncio tree

ba9fdb0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Nexus: worker, workflow-backed operations, and workflow caller #813

Nexus: worker, workflow-backed operations, and workflow caller #813

dandavison commented Apr 2, 2025 •

edited

Loading

Uh oh!

cretz left a comment

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025 •

edited

Loading

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025 •

edited

Loading

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025

Uh oh!

cretz Jun 27, 2025

Uh oh!

Uh oh!

		"""The task queue of the worker handling this Nexus operation."""


		def info() -> Info:

		# TODO(nexus-preview): How do we recommend that users create endpoints in their own tests?
		# See https://github.com/temporalio/sdk-typescript/pull/1708/files?show-viewed-files=true&file-filters%5B%5D=&w=0#r2082549085

		await _test_start_operation(test_case, True, env)


		async def _test_start_operation(

		# TODO(nexus-prerelease): do we need to track cancel operation
		# tasks as we do start operation tasks?

Nexus: worker, workflow-backed operations, and workflow caller #813

Are you sure you want to change the base?

Nexus: worker, workflow-backed operations, and workflow caller #813

Conversation

dandavison commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes for reviewers

Uh oh!

cretz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Jun 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dandavison commented Apr 2, 2025 •

edited

Loading

cretz Jun 27, 2025 •

edited

Loading

cretz Jun 27, 2025 •

edited

Loading